Image defect visibility predictor

ABSTRACT

In at least some examples, a system comprises a processor and a memory coupled to the processor. The memory stores an image defect visibility predictor that, when executed by the processor, compares an original image with a defect image and outputs a predicted defect visibility image (PDVI) that accounts for defect masking by the original image.

BACKGROUND

In commercial printing contexts, it is quite reasonable that customersexpect good print quality printed documents from a large scale high-endprinter, such as the HP Indigo Digital Press series. The HP IndigoDigital Press series of presses are used for general commercialprinting, including functions such as direct mail, publications, photo,flexible packaging, labels, and folding cartons. The HP Indigo DigitalPress series of presses can also used for specialty printing, since thisseries of presses can print without films and plates. Furthermore, theHP Indigo Digital Press series of presses have several embedded in linescanners, which can enable the operators to compare the scanned image tothe digital reference image on the fly. This function enables theoperators to observe print defects, then change images, text, and jobswithout stopping the press.

Due to customer expectations, print shops employing high-end printerneed to design their workflow to pay attention to quality. Thus, theissue of print quality assessment is quite important for developers ofcommercial printing systems. However, there are not many well-developedintegrated measure-ments of print quality.

BRIEF DESCRIPTION OF THE DRAWINGS

For a detailed description of illustrative examples, reference will nowbe made to the accompanying drawings in which:

FIG. 1 shows a computer system in accordance with various examples ofthe disclosure;

FIG. 2 shows a block diagram of system components and operations inaccordance with various examples of the disclosure;

FIG. 3 shows another block diagram of system components and operationsin accordance with various examples of the disclosure;

FIG. 4 shows a block diagram of system components and operations fortraining an image defect visibility predictor in accordance with variousexamples of the disclosure;

FIG. 5 shows a framework for an image defect visibility predictor inaccordance with various examples of the disclosure;

FIG. 6 shows a mechanical band measurement (MBM) overview in accordancewith various examples of the disclosure;

FIG. 7 shows a framework for training an image defect visibilitypredictor in accordance with various examples of the disclosure;

FIG. 8 shows a framework for testing an image defect visibilitypredictor in accordance with various examples of the disclosure;

FIG. 9 shows a method in accordance with various examples of thedisclosure;

FIG. 10 shows a screenshot of an original content image in accordancewith an example of the disclosure;

FIG. 11 shows a screenshot of the original content image of FIG. 10 withbanding defects in accordance with an example of the disclosure;

FIG. 12 shows a screenshot of a defect image related to the defects ofFIG. 11 in accordance with an example of the disclosure;

FIG. 13 shows a screenshot of a subject marked image related to defectsof FIG. 11 in accordance with an example of the disclosure;

FIG. 14 shows a screenshot of a ground truth image related to thedefects of FIG. 11 in accordance with an example of the disclosure;

FIG. 15 shows a screenshot of a modified ground truth image related tothe defects of FIG. 11 in accordance with an example of the disclosure;

FIG. 16 shows a screenshot of a predicted defect visibility image (PDVI)related to the defects of FIG. 11 in accordance with an example of thedisclosure;

FIG. 17 shows a screenshot of a raw mechanical band measurement (MBM)score chart in accordance with an example of the disclosure;

FIG. 18 shows a screenshot of a back projected MBM image in accordancewith an example of the disclosure;

FIG. 19 shows a screenshot of a modulated MBM image in accordance withan example of the disclosure;

FIG. 20 shows a screenshot of a PDVI result related to the defects ofFIG. 11 for a first quantization level in accordance with an example ofthe disclosure;

FIG. 21 shows a screenshot of a PDVI result related to the defects ofFIG. 11 for a second quantization level in accordance with an example ofthe disclosure;

FIG. 22 shows a screenshot of a visualization map related to the defectsof FIG. 11 for the second quantization level in accordance with anexample of the disclosure;

FIG. 23 shows a screenshot of a texture likelihood map for the originalcontent image of FIG. 10 in accordance with an example of thedisclosure; and

FIG. 24 shows another computer system in accordance with an example ofthe disclosure.

NOTATION AND NOMENCLATURE

Certain terms are used throughout the following description and claimsto refer to particular system components. As one skilled in the art willappreciate, computer companies may refer to a component by differentnames. This document does not intend to distinguish between componentsthat differ in name but not function. In the following discussion and inthe claims, the terms “including” and “comprising” are used in anopen-ended fashion, and thus should be interpreted to mean “including,but not limited to . . . . ” Also, the term “couple” or “couples” isintended to mean either an indirect, direct, optical or wirelesselectrical connection. Thus, if a first device couples to a seconddevice, that connection may be through a direct electrical connection,through an indirect electrical connection via other devices andconnections, through an optical electrical connection, or through awireless electrical connection.

DETAILED DESCRIPTION

Examples of the disclosure are directed to methods and systems for aMasking-Mediated Print Defect Visibility Predictor (MMPDVP) model orframework. Without limitation, the disclosed MMPDVP model is focused onthe print quality for real printed documents produced by large-scale andhigh-end printers and predict the visibility of defects in the presenceof customer content. In at least some examples, parameters of the MMPDVPmodel are trained from modified ground-truth images that have beenmarked by subjects. The output of the MMPDVP model (a predicted defectvisibility image or PDVI) may be used to help a press operator decidewhether the print quality is acceptable for specific customerrequirements. The output of the MMPDVP model can also be used tooptimize the print-shop workflow.

Typical documents printed commercially contain many images. Thissituation makes the images an important part in determining printquality. Images can be produced by many devices, such as monitors,printers, and copiers, although researchers usually focus on the imagequality and image fidelity which are not produced by printers but ratherthe monitors or the cameras. The existing image quality or fidelityassessment models are still a valuable area for investigation. Imagequality and image fidelity are not the same, but generally they are usedinterchangeably. As used herein, “image quality” refers to thepreference of one image over the others, while “image fidelity” refersto the accuracy between two images. Here they are considered together inthe same category, since most of the assessment models on image qualityand fidelity have the same purpose. Usually, one can describe the imagequality assessment assignments in the framework of image fidelity.

The disclosed MMPDVP model accepts two kinds of images as input: 1) acustomer's original digital content image; and 2) a customer's originaldigital content image with defects. Using these inputs, the MMPDVP modelwill generate an overall predicted map that shows where the viewer mightobserve a defect.

In at least some examples, MMPDVP model will take into account thecontent-masking effect of natural images which are produced by acommercial high-end printer. The MMPDVP model also may train itsparameters on modified ground truth images which are marked by subjectsin a psychophysical experiment. Furthermore, since banding is one of themost common print defects, the MMPDVP model targets banding artifactsand provides a final prediction map that estimates where the viewer willobserve banding.

FIG. 1 shows a computer system 100 in accordance with examples of thedisclosure. As shown, the computer system 100 comprises a processor 104in communication with a network interface 106 and a memory 108, wherethe memory 108 stores an image defect visibility predictor program 110.When executed by the processor 104, the image defect visibilitypredictor program 110, which comprises computer-readable instructions,compares an original content image with a defect image and outputs apredicted defect visibility image (PDVI) that accounts for defectmasking. In at least some examples, the defect image is based oncomparing the original content image with a scanned printout of theoriginal content image.

The image defect visibility predictor program 110, when executed, mayperform various operations to determine the PDVI for a customer'soriginal digital content image. For example, the image defect visibilitypredictor program 110 may determine a masking potential value and alightness value for the customer's digital original content image,determine a banding visibility value for the defect image, and outputthe PDVI based on the masking potential value, the lightness value, andthe banding visibility value. Additionally or alternatively, the imagedefect visibility predictor program 110 may determine a texture valueand/or a saliency value for the customer's original digital contentimage, and output the PDVI based on the texture value and/or thesaliency value. Further, the image defect visibility predictor program110, when executed, may determine a masking potential index image and alightness index image based on the customer's original digital contentimage, and a banding visibility index image based on the defect image.Similarly, the image defect visibility predictor program 110, whenexecuted, may determine a texture index image and a saliency indeximage.

In at least some examples, the image defect visibility predictor program110 employs a look-up table (LUT) to select the PDVI based on themasking potential index image, the lightness index image, the bandingvisibility index image, the texture index image, and/or the saliencyindex image. To determine the masking potential index image, the imagedefect visibility predictor program 110 may quantize a masking potentialimage that results from application of a local standard deviation to theoriginal content image. Further, the image defect visibility predictorprogram 110, when executed, may determine the lightness index image byquantizing lightness values detected for the original content image.Further, the image defect visibility predictor program 110, whenexecuted, may determine the banding visibility index image by quantizinga banding visibility image determined for the defect image. Further, theimage defect visibility predictor program 110, when executed, maydetermine the texture index image by quantizing a texture map or imagedetermined for the original content image. Further, the image defectvisibility predictor program 110, when executed, may determine thesaliency index image by quantizing a saliency map or image determinedfor the original content image. As will later be described in furtherdetail, the image defect visibility predictor program 110 may be trainedusing a set of ground truth images marked by human subjects.

FIG. 2 shows a block diagram 200 of system components and operations inaccordance with various examples of the disclosure. As shown in theblock diagram 200, an original content image 202 and stimuli 204 arecompared by comparison component 206 to generate a defect image 208. Thedefect image 208 is input to a MMPDVP 210, which outputs a correspondingPDVI 212.

FIG. 3 shows another block diagram 300 of system components andoperations in accordance with various examples of the disclosure. In theblock diagram 300, a portable document format (PDF) of an originalcontent image 302 is provided to a rasterizing image processor (RIP)304, which converts the PDF file into an original digital content image306. The original digital content image 306 is input to the digitalpress 308 to generate a printout 310. The printout 310 is scanned byin-line scanners 312 to provide a scanned image 314 with defects.

As shown, the customer's original digital content image 306 and thescanned image 314 are compared by comparison component 316, resulting indefect image 318. The MMPDVP 320 receives the original content image 306and the defect image 318 as input and outputs the PDVI 322, where thegray-scale levels of the PDVI 322 indicate the visibility of thedefects. More specifically, black in the PDVI 322 indicates a lowprobability that customers will observe a defect and white indicates ahigh probability that customers will observe a defect. Variousintermediate gray-scale values can be used also to show differentlikelihoods that customers will observe a defect. In the disclosed PDVIexamples, banding is the defect being analyzed and thus black indicatesa low probability that banding will be detected by consumers, whilewhite indicates that there is a high probability that banding will notbe detected by consumers. Other defects may additionally oralternatively be analyzed in addition to banding. Examples of defectsinclude streaks, spots, ghosting defects, and repetitive defects due tocontamination or damage to a rotating component.

FIG. 4 shows a block diagram 400 of system components and operations fortraining an image defect visibility predictor in accordance with variousexamples of the disclosure. In the block diagram 400, a customer'soriginal digital content image 404 is chosen and/or created at block402. Also, a banding characterization module 406 is used to determinethe banding prototype signal and banding features 408. The originaldigital content image 404 (sometimes referred to herein as C_(o)[m,n])and the banding prototype and features 408 are input to bandingsimulation block 410, which outputs an image with banding as a trainingstimulus 412. Repeating this process for different banding prototypesignals and different original digital content images results in a setof training stimuli. The training stimuli 412 are provided to humansubjects as part of a psychophysical experiment 414, which results in aset of subject marked images. Each such subject marked image 416(sometimes referred to herein as S[m,n]) indicates where banding isdetected. The subject marked image 416 is encoded to gray scale at block418, resulting in a ground truth image (GTI) 420 (sometimes referred toherein as G[m,n]). The GTI 420 is provided as an input to multiplier428. The other input of the multiplier 428 is obtained by comparing theoriginal digital content image 404 with the training stimuli (e.g., animage with banding) 412 using comparison component 422. The output ofthe comparison component 422 is defect image 424, which is scaled atblock 426 prior to being input to the multiplier 428.

The multiplier 428 operates to prevent banding marks in the GTI 420 thatare based on improper marking in the subject marked image 416 from beingpropagated. The output of the multiplier 428 is provided to quantizer430, which quantizes the values from the multiplier 428, resulting in amodified GTI 432. For example, the quantizer 430 may operate to assign avalue of 0, 1, 2, or 3 to the pixels or regions of the GTI 420. Thevalue 0 may correspond to areas that were not marked by a subject. Thevalue 1 may correspond to areas that were marked with a first color(e.g., green) that represents a low level of defects (e.g., banding).The value 2 may correspond to areas that were marked with a second color(e.g., yellow) that represents a medium level of defects. The value 3may correspond to areas that were marked with a third color (e.g., red)that represents a high level of defects.

As previously mentioned, banding is one of the most common printartifacts. It usually appears as a luminance variation and a chromaticvariation across a printed page in the scan direction, which isperpendicular to the paper process direction. FIG. 10 shows a screenshot1000 of an original content image in accordance with examples of thedisclosure. FIG. 11 shows a screenshot 1100 of the original contentimage of FIG. 10 with banding defects. FIG. 12 shows a screenshot 1200of a defect image (e.g., defect image 424) related to the defects ofFIG. 11 in accordance with an example of the disclosure. FIG. 13 shows ascreenshot 1300 of a subject marked image related to defects of FIG. 11in accordance with an example of the disclosure. FIG. 14 shows ascreenshot 1400 of a ground truth image (e.g., GTI 420) related to thedefects of FIG. 11 in accordance with an example of the disclosure. FIG.15 shows a screenshot 1500 of a modified ground truth image (e.g., MGTI432) related to the defects of FIG. 11 in accordance with an example ofthe disclosure.

FIG. 5 shows a framework 500 for an image defect visibility predictor inaccordance with various examples of the disclosure. The operation of theframework 500 may be performed, for example, by execution of the imagedefect visibility predictor 110 described previously. In framework 500,an original digital content image 502 and a defect image 522 areprovided as inputs and a PDVI look-up table (LUT) 550 is used to selector generate a PDVI 552. The inputs to the PDVI LUT 550 may correspond toa masking potential value, a lightness value, and a banding visibilityvalue. Additionally or alternatively, a texture value and/or a saliencyvalue may be inputs to the PDVI LUT 550 to select or generate PDVI 552.

To obtain the masking potential value, a local standard deviation 504 isapplied to the original digital content image 504, resulting in amasking potential image 506. A J-quantizer algorithm 508 is then appliedto the masking potential image 506, resulting in a masking potentialindex image 510. In at least some examples, the masking potential indeximage 510 corresponds to the masking potential value considered by thePDVI LUT 550 to determine PDVI 552.

To obtain the lightness value, an L-quantizer algorithm 512 is appliedto the original digital content image 502, resulting in a lightnessindex image 514. In at least some examples, the lightness index image514 corresponds to the lightness value considered by the PDVI LUT 550 todetermine PDVI 552.

To obtain the banding visibility value, the defect image 522 is providedto a mechanical band measurement (MBM) algorithm 524, resulting in a rawMBM score 526. FIG. 17 shows a screenshot 1700 of a raw MBM score chartrelated to a defect image in accordance with an example of thedisclosure. Further, FIG. 6 shows a mechanical band measurement (MBM)overview 600 in accordance with various examples of the disclosure. Asshown in the MBM overview 600, an digital press 602 generates a printoutof a test job 604. The printout 604 is scanned by scanner 606 and an MBMalgorithm 608 is applied to the scanned image, resulting in a 1-D score610.

Returning to FIG. 5, the raw MBM score 526 is input to the backprojection algorithm 528, resulting in a back projected MBM image 532.FIG. 18 shows a screenshot 1800 of a back projected MBM image inaccordance with an example of the disclosure. The back projected MBMimage 532 is input to a multiplier 534. The other input to themultiplier 534 corresponds to a scaled version (performed by scalingblock 532) of the defect image 522. The output of the multiplier 534corresponds to a banding visibility image 536 that is then provided to aK-quantizer algorithm 538, resulting in a banding visibility index image540. FIG. 19 shows a screenshot 1900 of a modulated MBM image (referredherein as B[m,n]) in accordance with an example of the disclosure. Themodulated MBM image of screenshot 1900 corresponds to the bandingvisibility image 536 output from the multiplier 534. In at least someexamples, the banding visibility index image 540 corresponds to thebanding visibility value considered by the PDVI LUT 550 to determinePDVI 552.

FIG. 16 shows a screenshot 1600 of a PDVI (e.g., PDVI 552) related tothe defects of FIG. 11 in accordance with an example of the disclosure.The PDVI of screenshot 1600 is based on a quantization level whereL=K=J=32. FIG. 20 shows a screenshot 2000 of a PDVI (e.g., PDVI 552)related to the defects of FIG. 11 in accordance with an example of thedisclosure. The PDVI of screenshot 2000 is based on a quantization levelwhere L=K=J=16 and where text areas of the image are noted since theyhave different banding visibility characteristics compared to othertypes of image content. FIG. 21 shows a screenshot 2100 of a PDVI (e.g.,PDVI 552) related to the defects of FIG. 11 in accordance with anexample of the disclosure. The PDVI of screenshot 2100 is based on aquantization level where L=K=J=32 (the same quantization as thescreenshot 1600 of FIG. 16) and where text areas of the image are notedsince they have different banding visibility characteristics compared toother types of image content. Comparing the noted text areas in FIG. 20and FIG. 21, FIG. 21 shows a more accurate result, since the PDVI hasless response (black color) to indicate low banding visibilityprediction in those noted areas, where the subject didn't mark in FIG.13. The PDVI result in FIG. 21 also shows high response (white color) toindicate high banding visibility prediction in the noted areas shown inFIG. 13, where the subject observes strong banding.

FIG. 22 shows a screenshot 2200 of a visualization map for quantizationlevel L=K=J=32. The visualization map of screenshot 2200 is used to testand illustrate the MMPDVP model. In the visualization map, a uniquevalue to each j, k and l level is assigned respectively into the R, G, Bchannel. In this manner, it can be verified that each J, K, and Lcombination has a different fixed color in this map, which means eachcombination is independent from the others.

As shown in FIG. 5, additional parameter values may be input to the PDVILUT 550. For example, a texture value could additionally oralternatively be input to the PDVI LUT 550. The texture value may dedetermined, for example, by detecting texture from an original contentimage, resulting in a texture likelihood image or map (T[m,n]). FIG. 23shows a screenshot 2300 of a texture likelihood map for the originalcontent image of FIG. 10 in accordance with an example of thedisclosure. The texture likelihood map of screenshot 2300 may be used todetermine a texture value that is one basis for determining PDVI 552. Aswith the other feature inputs described herein, a texture likelihood mapmay be quantized and a texture likelihood index image (t[m,n]) maycorrespond to the texture value provided as input to the PDVI LUT 550.

Another parameter value that may additionally or alternatively be inputto the PDVI LUT 550 is a saliency value. The saliency value may bedetermined, for example, by pre-processing the original content image toidentify saliency-objects. Identification of saliency-objects is used tobuild a saliency-object map (S_(o)[m,n]). As with the other featureinputs described herein, a saliency-object map may be quantized and asaliency-object index image (S_(o)[m,n]) may correspond to the saliencyvalue provided as input to the PDVI LUT 550. In at least some examples,the saliency value should provide sharp boundaries for saliency-objects.

FIG. 7 shows a framework 700 for training an image defect visibilitypredictor in accordance with various examples of the disclosure. Inframework 700 a modified GTI 702 is input to a cost function 714. Thecost function 714 also receives as input a PDVI 712 that is output froma MMPDVP 708 based on an original digital content 704 and a defect image706. As shown, the MMPDVP 708 uses fixed architecture, fixed features,and free parameters at block 710, which may be updated by an optimalparameter selection block 716 that receives an output of the costfunction 714. In accordance with some examples, the cost function 714penalizes the difference between the output PDVI of the MMPDVP and themodified ground truth data. This cost function is minimized by optimalparameter selection 716 to obtain optimized parameters at block 710 forthe predictor 708.

FIG. 8 shows a framework 800 for offline testing of an image defectvisibility predictor in accordance with various examples of thedisclosure. The framework 800 includes a testing process 809 that usesan MMPDVP 810 with fixed architecture, fixed features, and fixedparameters at block 812 to provide input to a cost function 824. At thisstage, the free parameters of block 710 have been chosen by optimalparameter selection 716, and are now fixed at block 812. The costfunction 824 also receives modified ground truth images 822 as marked bysubjects as input. Over time, the MMPDVP 810 may output a plurality ofPDVIs to the cost function 824. Each PDVI is based on a given defectimage from defect images 808 and a given original content image fromoriginal content images 802. As shown, the defect images 808 are theresult of comparing (using comparison component 806) original contentimages 802 with stimuli from testing stimuli set 804. The testingstimuli set 804 is also used for psychophysical experiments 814,resulting in ground truth images 816 as marked by subjects. The modifiedground truth images 822 that are input to the cost function 824 areobtained by rescaling and quantizing the ground truth images 816 atrescaling block 818 and quantizing block 820.

FIG. 9 shows a method 900 in accordance with various examples of thedisclosure. The method 900 may be performed, for example, by a processorof a computer system such as computer system 100. As shown, the method900 comprises receiving an original content image at block 902 andreceiving a defect image at block 904. The original content image andthe defect image may be obtained as described herein. At block 906, aPDVI that accounts for defect masking is determined based on comparisonof the original content image and the defect image.

In FIG. 5, there are three features that are quantized into certainlevels to obtain the three classification index images. The threequantized feature images are: 1) the masking potential index image j[m,n]; 2) the lightness index image l[m, n]; and 3) the banding visibilityindex image k[m, n]. Additional features such as texture, saliency, andfacial recognition may be quantized as well. Further, it should be notedthat all the input images described herein are gray-scale images. In atleast some examples, an image is converted from sRGB space into CIE XYZspace, then is further converted from CIE XYZ space into L*a*b* space.Finally, the L* channel is used as the gray-scale image input.

The quantizers disclosed herein have the same structure. As describedherein, the masking potential index image j[m, n], lightness index imagel[m, n], and banding visibility (MBM) index image k[m, n] are obtainedfrom the masking potential image M[m, n], the original digital contentimage C_(o)[m, n], and the modulated MBM B[m, n], respectively. For themasking potential index image j[m, n], the local standard deviation fromthe customer's original digital content image C_(o)[m, n] is used insome examples to obtain the masking potential image M[m, n]. The maskingpotential image M[m, n] can provide the information about how the imagecontent masks the defect. Then M[m, n] is quantized by a certain levelquantizer to obtain the masking potential index image j[m, n].

The mathematical description of the J-quantizer is:

$\begin{matrix}\begin{matrix}{{j\left\lbrack {m,n} \right\rbrack} = {Q^{(M)}\left( {M\left\lbrack {m,n} \right\rbrack} \right)}} \\{= \left\{ {\begin{matrix}{\left\lfloor {\overset{\Cup}{M}\left\lbrack {m,n} \right\rbrack} \right\rfloor,{0 \leq {\overset{\Cup}{M}\left\lbrack {m,n} \right\rbrack} < J}} \\{{J - 1},{{\overset{\Cup}{M}\left\lbrack {m,n} \right\rbrack} = J}}\end{matrix},} \right.}\end{matrix} & (1.1) \\{{{\overset{\Cup}{M}\left\lbrack {m,n} \right\rbrack} = {J\frac{{M\left\lbrack {m,n} \right\rbrack} - {\min\left( {M\left\lbrack {m,n} \right\rbrack} \right)}}{{\max\left( {M\left\lbrack {m,n} \right\rbrack} \right)} - {\min\left( {M\left\lbrack {m,n} \right\rbrack} \right)}}}},} & (1.2)\end{matrix}$where J is the total number of quantized levels for M[m, n]. Equation(1.1) is the definition of J-quantizer. The purpose of equation (1.2) isto rescale M[m, n] into the interval [0, J]. Then the pixel values inthe rescaled image M[m, n] are quantized according to equation (1.1),where └x┘ denotes flooring x to the nearest integer that is less than orequal to x. Furthermore, when rescaled {hacek over (M)}[m,n] has thevalue J, it is converted to J−1 as in equation (1.1).

For the lightness index image l[m,n], original gray scale image isquantized which is the L⁺ channel in L⁺a⁺b⁺space to obtain the lightnessindex image l[m, n]. The definition of the L-quantizer is:

$\begin{matrix}\begin{matrix}{{l\left\lbrack {m,n} \right\rbrack} = {Q^{(C_{o})}\left( {C_{o}\left\lbrack {m,n} \right\rbrack} \right)}} \\{= \left\{ \begin{matrix}{\left\lfloor {\overset{\Cup}{C_{o}}\left\lbrack {m,n} \right\rbrack} \right\rfloor,{0 \leq {\overset{\Cup}{C_{o}}\left\lbrack {m,n} \right\rbrack} < L}} \\{{L - 1},{{\overset{\Cup}{C_{o}}\left\lbrack {m,n} \right\rbrack} = L}}\end{matrix} \right.}\end{matrix} & (1.3) \\{{\overset{\Cup}{C_{0}}\left\lbrack {m,n} \right\rbrack} = {L\frac{{C_{0}\left\lbrack {m,n} \right\rbrack} - {\min\left( {C_{0}\left\lbrack {m,n} \right\rbrack} \right)}}{{\max\left( {C_{0}\left\lbrack {m,n} \right\rbrack} \right)} - {\min\left( {C_{0}\left\lbrack {m,n} \right\rbrack} \right)}}}} & (1.4)\end{matrix}$This mathematical description is similar to the definition ofJ-quantizer, where L is the total number of quantized levels forC_(o)[m, n]. Equation (1.3) is the definition of L-quantizer. Thenequation (1.4) is used to rescale C_(o)[m, n] into interval [0, L].].The pixel values in the rescaled image {hacek over (C)}₀[m,n] arequantized according to equation (1.3).

For the banding visibility index image k[m, n], the defect image D[m, n]from the original digital content image C_(o)[m, n] is obtained bysubtracting stimuli image C_(b)[m, n]. Then D[m, n] is taken as theinput to the Mechanical Band Measurement (MBM) tool. As describedherein, the raw MBM score R[n] (1−D) is back projected to obtain a 2-Dimage referred to herein as a back projected MBM image M_(b)[m, n],which has constant banding all along the vertical direction. M_(b)[m, n]is then multiplied by rescaled D[m, n] to obtain a modulated MBM imageB[m, n], which predicts how the subjects will see a defect in agray-scale image. By multiplying M_(b)[m, n] by the rescaled D[m, n],the defect image modulated MBM B[m, n] can accurately depict thecharacter of the defect in terms of what the subjects observe.

Using the same technique, the banding visibility index image k[m,n] maybe determined. In at least some examples, the K-quantizers are definedas:

$\begin{matrix}\begin{matrix}{{k\left\lbrack {m,n} \right\rbrack} = {Q^{(B)}\left( {\overset{\Cup}{B}\left\lbrack {m,n} \right\rbrack} \right)}} \\{= \left\{ {\begin{matrix}{\left\lfloor {\overset{\Cup}{B}\left\lbrack {m,n} \right\rbrack} \right\rfloor,{0 \leq {\overset{\Cup}{B}\left\lbrack {m,n} \right\rbrack} < L}} \\{{K - 1},{{\overset{\Cup}{B}\left\lbrack {m,n} \right\rbrack} = L}}\end{matrix},} \right.}\end{matrix} & (1.5) \\{{{B\left\lbrack {m,n} \right\rbrack} = {{M_{b}\left\lbrack {m,n} \right\rbrack}\frac{D\left\lbrack {m,n} \right\rbrack}{\max\left( {D\left\lbrack {m,n} \right\rbrack} \right)}}},} & (1.6) \\{{{\overset{\Cup}{B}\left\lbrack {m,n} \right\rbrack} = {K\frac{{B\left\lbrack {m,n} \right\rbrack} - {\min\left( {B\left\lbrack {m,n} \right\rbrack} \right)}}{{\max\left( {B\left\lbrack {m,n} \right\rbrack} \right)} - {\min\left( {B\left\lbrack {m,n} \right\rbrack} \right)}}}},} & (1.7)\end{matrix}$where K is the total number of quantized levels for B[m, n]. Equation(1.5) is the definition of K-quantizer. Then equation (1.6) defines themodulated MBM image, B[m,n]. Equation (1.7) is used to rescale B[m,n]into interval [0,K]. The pixel values in the rescaled image B[m,n] arequantized according to equation (1.5).

In accordance with examples of the disclosure, the masking potentialimage will account for the image content masking effect, the lightnessimage will account for the lightness dependence of defect visibility,and the defect visibility image will provide the defect information. Theimpact of these three features on overall defect visibility issummarized by the three quantized index images, which are analogous tosegmentation images. The predicted defect visibility is chosenindependently for each different combination of quantizer output levels.For each such combination, the predicted defect visibility is stored asa parameter in the PDVI LUT. By training these quantized images to themodified ground truth information, the parameters can be optimized, andbetter predict how the subjects observe the defects in a specificregion.

In at least some examples, the predictor is simply a 3-D LUT that yieldsan identical prediction for all occurrences of the same three-tuple ofvalues from the three index images. To specify its structure, thedifferent regions of each quantization level are defined in our indeximages according to:Ω_(j) ₀ ^((M)) ={[m,n]:j[m,n]=j ₀},0≦j ₀ ≦J−1  (1.8).Ω_(l) ₀ ^((C) ⁰ ⁾ ={[m,n]:l[m,n]=l ₀},0≦l ₀ ≦L−1  (1.9).Ω_(k) ₀ ^((B)) ={[m,n]:k[m,n]=k ₀},0≦k ₀ ≦K−1  (1.10).These three equations are similar. All the pixels with the same quantizelevel are in the same segment region. The definition of these regionswill be used for the training and testing process. The PDVI result isdefined as:Ĝ[m,n]=ε ^((MBC) ⁰ ⁾ [j[m,n],k[m,n],l[m,n]]  (1.11).Equation (1.11) provides the mapping that will be trained on themodified ground truth images.

Once conversion of the subject marked image S[m, n] to the ModifiedGround Truth Image (MGTI) G_(M)[m, n] is finished, then training theparameters on the modified ground truth information is performed toobtain the optimized parameters for the MMPDVP. After generating theoriginal digital content image and the defect image, these two imagesare used as input to the MMPDVP with free parameters. Then the costfunction is calculated, which penalizes the difference between theoutput PDVI of the MMPDVP and the modified ground truth data.

The cost function in a simple form is defined as following:

$\begin{matrix}{\phi = {\sum\limits_{j = 0}^{J - 1}{\sum\limits_{k = 0}^{K - 1}{\sum\limits_{l = 0}^{L - 1}\underset{{({m,n})} \in {\Omega_{j}^{(M)}\bigcap\Omega_{k}^{(B)}\bigcap\Omega_{l}^{(C_{0})}}}{\overset{\;}{\;}}{{{{\hat{G}\left\lbrack {m,n} \right\rbrack} - {G_{M}\left\lbrack {m,n} \right\rbrack}}}^{2}.}}}}} & (2.1)\end{matrix}$Here the regions Ω_(j) ₀ ^((M)), Ω_(l) ₀ ^((C) ⁰ ⁾, Ω_(k) ₀ ^((B)) aredefined by equation (1.8), (1.9), and (1.10). The image Ĝ[m, n] is thepredicted defect visibility image (PDVI); and the image G_(M)[m, n] isthe modified ground truth image (MGTI). For each region with a differentj, k, and l combination, the mean square error between the MGTI and thePDVI is calculated. In this case, a closed form for the optimal PDVI Foreach region with a different j, k, and l combination, the mean squareerror between the MGTI and the PDVI is calculated. In this case, aclosed form for the optimal parameters ò_(OPT) ^((MBC) ^(o) ⁾[j,k,l] isused. By minimizing the cost function, the optimized prediction isdefined as:

$\begin{matrix}{{ò_{OPT}^{({MBC}_{o})}\left\lbrack {j,k,l} \right\rbrack} = {\frac{\sum\limits_{{\lbrack{m,n}\rbrack} \in {\Omega_{j}^{(M)}\bigcap\Omega_{k}^{(B)}\bigcap\Omega_{l}^{(C_{0})}}}^{\;}{G_{M}\left\lbrack {m,n} \right\rbrack}}{\sum\limits_{{\lbrack{m,n}\rbrack} \in {\Omega_{j}^{(M)}\bigcap\Omega_{k}^{(B)}\bigcap\Omega_{l}^{(C_{0})}}}^{\;}1}.}} & (2.2)\end{matrix}$This is the mean of the image G_(M)[m, n] conditioned on the pixelvalues of the three index images being (j,k,l).In one example of the algorithm, the parameters are trained on multipleimage sets, each such set comprising a modified ground truth image 702,an original digital content image 704, and a defect image 706, asillustrated in FIG. 7. In this case, the cost function will be:

$\begin{matrix}{{\phi = {\sum\limits_{i = 0}^{I - 1}{\sum\limits_{j = 0}^{J - 1}{\sum\limits_{k = 0}^{K - 1}{\sum\limits_{l = 0}^{L - 1}{\sum\limits_{{\lbrack{m,n}\rbrack} \in^{I}\Omega_{j,k,l}^{({MBC}_{0})}}^{\;}{^{I}{{\hat{G}\left\lbrack {m,n} \right\rbrack} -^{I}{G_{M}\left\lbrack {m,n} \right\rbrack}}}^{2}}}}}}},} & (2.3)\end{matrix}$as the total squared error between the predicted and actual groundtruth. Here the parameter i indexes the image sets 702, 704, and 706used for the training, and l is the total number of such image sets usedfor training. For each region with a different combination of values forj, k, and l, the total squared error is calculated between the MGTI andthe PDVI. By minimizing the cost function, the optimized parametersò_(OPT) ^((MBC) ⁰ ⁾[j,k,l] are obtained as

$\begin{matrix}{{ò_{OPT}^{({MBC}_{o})}\left\lbrack {j,k,l} \right\rbrack} = {\frac{\sum\limits_{I = 0}^{I - 1}{{\sum\limits_{{\lbrack{m,n}\rbrack} \in^{I}\Omega_{j,k,l}^{({MBC}_{0})}}^{\;}}^{i}{G_{M}\left\lbrack {m,n} \right\rbrack}}}{\sum\limits_{I = 0}^{I - 1}{\sum\limits_{{\lbrack{m,n}\rbrack} \in^{I}\Omega_{j,k,l}^{({MBC}_{0})}}^{\;}1}}.}} & (2.4)\end{matrix}$which is the conditional mean of G_(M)[m, n], given the three-tuplevalue [j; k; l] for the three index images. These optimized parametersò_(OPT) ^(MBC) ⁰ ⁾[j,k,l] are stored in the PDVI LUT.

As previously discussed, texture and saliency may additionally oralternatively be used to identify a PDVI. The textured areas in naturalimages may be detected, for example, using an indicator based oncomponent counts. Further, a Gabor filter may be employed for texturedetection or segmentation. Further, face detection may be used toidentify important regions that would be given a higher weighting fordefect visibility in computing the PDVI. Further, a filter bank forimage segmentation and classification may be used. Further, the MMDVPmodel may be changed to a classifier such as a Gaussian Mixture Model(GMM) or a Support Vector Machine (SVM). Further, the scope of thisinvention is not limited to banding as the only defect. There are manyother printing issues, such as oil spots and unexpected marks on theprint. Unlike banding defects, some of these types of defects may bemore noticeable on a face or main areas of an image. Therefore, in someexamples, face detection and a saliency map may be additional featuresto address these types of defects. Finally, operations such asexamination of the values for the error metric after optimization,investigating the effectiveness of training, and cross-validating may beperformed to test updates to the model.

The MMPDVP techniques as disclosed above may be implemented with anygeneral-purpose computing component, such as an application-specificintegrated chip (ASIC), a computer, or a network component withsufficient processing power, memory resources, and network throughputcapability to handle the necessary workload placed upon it. FIG. 24illustrates a typical, general-purpose computer system 2400 suitable forimplementing one or more examples of the components disclosed herein.The computer system 2400 includes a processor 2402 (which may bereferred to as a central processor unit or CPU) that is in communicationwith memory devices including secondary storage 2404, read only memory(ROM) 2406, and random access memory (RAM) 2408, with an input/output(I/O) interface 2410, and with a network interface 2412. The processor2402 may be implemented as one or more CPU chips, or may be part of oneor more application specific integrated circuits (ASICs).

The secondary storage 2404 is typically comprised of one or more diskdrives, flash devices, or tape drives and is used for non-volatilestorage of data and as an over-flow data storage device if RAM 2408 isnot large enough to hold all working data. Secondary storage 2404 may beused to store programs that are loaded into RAM 2408 when such programsare selected for execution. The ROM 2406 is used to store instructionsand perhaps data that are read during program execution. ROM 2406 is anon-volatile memory device that typically has a small memory capacityrelative to the larger memory capacity of secondary storage 2404. TheRAM 2408 is used to store volatile data and perhaps to storeinstructions. Access to both ROM 2406 and RAM 2408 is typically fasterthan to secondary storage 2404. The RAM 2408, the ROM 2406, the secondstorage 2404, and the memory 108 of FIG. 1 are examples ofnon-transitory computer-readable media.

The above discussion is meant to be illustrative of the principles andvarious examples of the present invention. Numerous variations andmodifications will become apparent to those skilled in the art once theabove disclosure is fully appreciated. It is intended that the followingclaims be interpreted to embrace all such variations and modifications.

What is claimed is:
 1. A system comprising: a processor; a memorycoupled to said processor; and wherein the memory stores an image defectvisibility predictor that, when executed by the processor, compares anoriginal image with a defect image and outputs a predicted defectvisibility image (PDVI) that accounts for defect masking by the originalimage, wherein the PDVI is based on a masking potential value thatprovides information about how content of the original image masks adefect in the defect image, wherein the image defect visibilitypredictor employs a look-up table (LUT) to select the PDVI based on amasking potential index image, a lightness index image, and a bandingvisibility index image, wherein the image defect visibility predictordetermines the masking potential index image by quantizing a maskingpotential image that results from application of a local standarddeviation to the original image.
 2. The system of claim 1, wherein thedefect image is based on comparing the original image with a scannedprintout of the original image.
 3. The system of claim 1, wherein theimage defect visibility predictor determines the masking potential valueand a lightness value for the original image, determines a bandingvisibility value for the defect image, and outputs the predicted defectvisibility image based on the masking potential value, the lightnessvalue, and the banding visibility value.
 4. The system of claim 1,wherein the image defect visibility predictor determines a texture mapbased on the original image and selects the PDVI based on the texturemap.
 5. The system of claim 1, wherein the image defect visibilitypredictor determines a saliency-object map based on the original imageand selects the PDVI based on the saliency-object map.
 6. The system ofclaim 1, wherein the image defect visibility predictor determines thelightness index image by quantizing lightness values detected for theoriginal image and determines the banding visibility index image byquantizing a banding visibility image determined for the defect image.7. The system of claim 1, wherein the image defect visibility predictoris trained using a set of ground truth images marked by human subjects.8. A non-transitory computer-readable medium that stores an image defectvisibility predictor program that, when executed, causes a processor tooutput a predicted defect visibility image (PDVI) that accounts fordefect masking based on comparison of an original image with a defectimage, wherein the PDVI is based on a masking potential value thatprovides information about how content of the original image masks adefect in the defect image.
 9. The non-transitory computer-readablemedium of claim 8, wherein the image defect visibility predictor programfurther causes the processor to compare the original image with ascanned printout of the original image to determine the defect image.10. The non-transitory computer-readable medium of claim 8, wherein theimage defect visibility predictor program further causes the processorto determine the masking potential value and a lightness value for theoriginal image, to determine a banding visibility value for the defectimage, and to output the PDVI based on the masking potential value, thelightness value, and the banding visibility value.
 11. Thenon-transitory computer-readable medium of claim 8, wherein the imagedefect visibility predictor program further causes the processor todetermine a masking potential index image and a lightness index imagebased on the original image, and to determine a banding visibility indeximage based on the defect image.
 12. The non-transitorycomputer-readable medium of claim 11, wherein the image defectvisibility predictor program further causes the processor to access alook-up table (LUT) to select the PDVI based on the masking potentialindex image, the lightness index image, and the banding visibility indeximage.
 13. The non-transitory computer-readable medium of claim 11,wherein the image defect visibility predictor program further causes theprocessor to determine the masking potential index image, the lightnessindex image, and the banding visibility index image by quantizingmasking potential values, lightness values, and banding visibilityvalues.
 14. A method comprising: receiving, by a processor, an originalimage; receiving, by the processor, a defect image; and determining, bythe processor, a predicted defect visibility image (PDVI) that accountsfor defect masking based on comparison of the original image and thedefect image; and outputting, by the processor, the PDVI that accountsfor defect masking by the original image, wherein the PDVI isgraphically coded to indicate a probability value that a defect in thedefect image will be detectable.
 15. The method of claim 14 furthercomprising determining the defect image by comparison of the originalimage with a scanned printout of the original image.
 16. The method ofclaim 14 further comprising determining a masking potential value and alightness value for the original content image, determining a bandingvisibility value for the defect image, and determining the PDVI based onthe masking potential value, the lightness value, and the bandingvisibility value.
 17. The method of claim 16, wherein determining thePDVI comprises accessing a look-up table (LUT) and selecting the PDVIbased on the masking potential value, the lightness value, and thebanding visibility value.
 18. The method of claim 16, further comprisingtraining an image defect visibility predictor algorithm based on a setof ground truth images marked by human subjects and determining the PDVIusing the trained image defect visibility predictor algorithm.